Using the EM Algorithm to Train Neural Networks: Misconceptions and a New Algorithm for Multiclass Classification

The expectation-maximization (EM) algorithm has been of considerable interest in recent years as the basis for various algorithms in application areas of neural networks such as pattern recognition. However, there exists some misconceptions concerning its application to neural networks. In this paper, we clarify these misconceptions and consider how the EM algorithm can be adopted to train multilayer perceptron (MLP) and mixture of experts (ME) networks in applications to multiclass classification. We identify some situations where the application of the EM algorithm to train MLP networks may be of limited value and discuss some ways of handling the difficulties. For ME networks, it is reported in the literature that networks trained by the EM algorithm using iteratively reweighted least squares (IRLS) algorithm in the inner loop of the M-step, often performed poorly in multiclass classification. However, we found that the convergence of the IRLS algorithm is stable and that the log likelihood is monotonic increasing when a learning rate smaller than one is adopted. Also, we propose the use of an expectation-conditional maximization (ECM) algorithm to train ME networks. Its performance is demonstrated to be superior to the IRLS algorithm on some simulated and real data sets.

Method for the generation and cultivation of functional three-dimensional mammary constructs without exogenous extracellular matrix

During puberty, pregnancy, lactation and postlactation, breast tissue undergoes extensive remodelling and the disruption of these events can lead to cancer. In vitro studies of mammary tissue and its malignant transformation regularly employ mammary epithelial cells cultivated on matrigel or floating collagen rafts. In these cultures, mammary epithelial cells assemble into three-dimensional structures resembling in vivo acini. We present a novel technique for generating functional mammary constructs without the use of matrix substitutes.

Pioglitazone inhibits cell growth and reduces matrix production in human kidney fibroblasts

Peroxisome proliferator-activated receptor-gamma (PPAR-gamma) agonists are increasingly used in patients with diabetes, and small studies have suggested a beneficial effect on renal function, but their effects on. extracellular matrix (ECM) turnover are unknown. The aims of this study were to investigate the effects of the PPAR-gamma agonist pioglitazone on growth and matrix production in human cortical fibroblasts (CF). Cell growth and ECM production and turnover were measured in human CF in the presence and absence of 1 and 3 muM pioglitazone. Exposure of CF to pioglitazone caused an antiproliferative (P < 0.0001) and hypertrophic (P < 0.0001) effect; reduced type IV collagen secretion (P < 0.01), fibronectin secretion (P < 0.0001), and proline incorporation (P < 0.0001); decreased MMP-9 activity (P < 0.05); and reduced tissue inhibitor of metalloproteinase-1 (TIMP-1) and TIMP-2 secretion (P < 0.001 and P < 0.0001, respectively). These effects were independent of TGF-beta1. A reduction in ECM production was similarly observed when CF were exposed to a selective PPAR-gamma agonist (L-805645) in concentrations that caused no toxicity, confirming the antifibrotic effects of pioglitazone were mediated through a PPAR-gamma-dependent mechanism. Exposure of CF to high glucose conditions induced an increase in the expression of collagen IV (P < 0.05), which was reversed both in the presence of pioglitazone (1 and 3 muM) and by L-805645. In summary, exposure of human CIF to pioglitazone causes an antiproliferative effect and reduces ECM production through mechanisms that include reducing TIMP activity, independent of TGF-beta1. These studies suggest that the PPAR-gamma agonists may have a specific role in ameliorating the course of progressive tubulointerstitial fibrosis under both normoglycemic and hyperglycemic states.

Novel markers for poor prognosis in head and neck cancer

Head and neck cancer (HNSCC) is one of the most distressing human cancers, causing pain and affecting the basic survival functions of breathing and swallowing. Mortality rates have not changed despite recent advances in radiotherapy and surgical treatment. We have compared the expression of over 13,000 unique genes in 7 cases of matched HNSCC and normal oral mucosa. Of the 1,260 genes that showed statistically significant differences in expression between normal and tumor tissue at the mRNA level, the three top ranking of the top 5% were selected for further analysis by immunohistochemistry on paraffin sections,. along with the tumor suppressor genes p16 and p53, in a total of 62 patients including 55 for whom >4-year clinical data was available. Using univariate and multivariate survival analysis, we identified SPARC/osteonectin as a powerful independent prognostic marker for short disease-free interval (DFI) (p < 0.002) and poor overall survival (OS) (p = 0.018) of HNSCC patients. In combination with other ECM proteins found in our analysis, PAI-1 and uPA, the association with DFI and OS became even more significant (p < 0.001). Our study represents the first instance of SPARC as an independent prognostic marker in HNSCC.

An Incremental EM-based Learning Approach for On-Line Prediction of Hospital Resource Utilization

Objective: Inpatient length of stay (LOS) is an important measure of hospital activity, health care resource consumption, and patient acuity. This research work aims at developing an incremental expectation maximization (EM) based learning approach on mixture of experts (ME) system for on-line prediction of LOS. The use of a batchmode learning process in most existing artificial neural networks to predict LOS is unrealistic, as the data become available over time and their pattern change dynamically. In contrast, an on-line process is capable of providing an output whenever a new datum becomes available. This on-the-spot information is therefore more useful and practical for making decisions, especially when one deals with a tremendous amount of data. Methods and material: The proposed approach is illustrated using a real example of gastroenteritis LOS data. The data set was extracted from a retrospective cohort study on all infants born in 1995-1997 and their subsequent admissions for gastroenteritis. The total number of admissions in this data set was n = 692. Linked hospitalization records of the cohort were retrieved retrospectively to derive the outcome measure, patient demographics, and associated co-morbidities information. A comparative study of the incremental learning and the batch-mode learning algorithms is considered. The performances of the learning algorithms are compared based on the mean absolute difference (MAD) between the predictions and the actual LOS, and the proportion of predictions with MAD < 1 day (Prop(MAD < 1)). The significance of the comparison is assessed through a regression analysis. Results: The incremental learning algorithm provides better on-line prediction of LOS when the system has gained sufficient training from more examples (MAD = 1.77 days and Prop(MAD < 1) = 54.3%), compared to that using the batch-mode learning. The regression analysis indicates a significant decrease of MAD (p-value = 0.063) and a significant (p-value = 0.044) increase of Prop(MAD

Relationship of body condition score and blood urea and ammonia to pregnancy in Italian Mediterranean buffaloes

The relationship of body condition score ( BCS) and blood urea and ammonia to pregnancy outcome was examined in Italian Mediterranean Buffalo cows mated by AI. The study was conducted on 150 buffaloes at 145 +/- 83 days in milk that were fed a diet comprising 14.8% crude protein, 0.9 milk forage units . kg(-1) dry matter and a non- structural carbohydrate/ crude protein ratio of 2.14. The stage of the oestrous cycle was synchronised by the Ovsynch- TAI programme and blood urea and ammonia levels were assessed on the day of AI. Energy corrected milk ( ECM) production and BCS were recorded bi- weekly. The pregnancy risk was 46.7% and was slightly lower in buffaloes with BCS < 6.0 and BCS > 7.5. There were no significant differences in ECM, urea and ammonia between pregnant and non- pregnant buffaloes. However, pregnancy outcome was higher ( P = 0.02) in buffaloes with blood urea < 6.83 mmol . L-1. The likelihood of pregnancy for buffaloes with low urea blood level was 2.6 greater than for high urea level and exposure to a high urea level lowered the probability of pregnancy by about 0.25. The findings indicate that buffaloes are similar to cattle and increased blood levels of urea are associated with reduced fertility when animals are mated by AI.

The renal cortical fibroblast in renal tubulointerstitial fibrosis

Renal cortical fibroblasts have key roles in mediating intercellular communication with neighboring/infiltrating cells and extracellular matrix (ECM) and maintenance of renal tissue architecture. They express a variety of cytokines, chemokines, growth factors and cell adhesion molecules, playing an active role in paracrine and autocrine interactions and regulating both fibrogenesis and the interstitial inflammatory response. They additionally have an endocrine function in the production of epoetin. Tubulointerstitial fibrosis, the common pathological consequence of renal injury, is characterized by the accumulation of extracellular matrix largely due to excessive production in parallel with reduced degradation, and activated fibroblasts characterized by a myofibroblastic phenotype. Fibroblasts in the kidney may derive from resident fibroblasts, from the circulating fibroblast population or from haemopoetic progenitor or stromal cells derived from the bone marrow. Cells exhibiting a myofibroblastic phenotype may derive from these sources and from tubular cells undergoing epithelial to mesenchymal transformation in response to renal injury. The number of interstitial myofibroblasts correlates closely with tubulointerstitial fibrosis and progressive renal failure. Hence inhibiting myofibroblast formation may be an effective strategy in attenuating the development of renal failure in kidney disease of diverse etiology. (c) 2005 Elsevier Ltd. All rights reserved.